Skip to main content

Roles and Access Control for API Keys

API Keys​

API keys can be used not only for authentication, but also to manage access to system resources and apply restrictions on such access.

This is done via assigning of roles to API keys. Roles must be associated with specific system resources such as models, applications, addons and assistants.

To create and configure access control for API keys:

  1. Create API Key

  2. Assign role(s)

  3. Add restrictions:

    Roles used by API keys must be declared in the roles section. In the following example, we assign the basic role for projectKey1 API key. It grants access to the chat-gpt-35-turbo model with some restrictions:

    //Example extract from aidial.config.json
    "keys": {
        "proxyKey1": { //API key
            "project": "Project1",
            "role": "basic" // the name of the role
        }
    },
    "roles": {
        "basic": { // the name of the role
            "limits": {
                "chat-gpt-35-turbo": {
                "minute": "100000", //number of tokens per minute
                "day": "10000000", //number of tokens per day
                "week": "10000000", //number of tokens per week
                "month": "10000000", //number of tokens per month
                }
            }
        }
    }

    In settings, you can get familiar with the description of the configuration parameters. Refer to configuration file to view the full example.

Per-Request Keys​

Per-request keys are used to manage access to user files for applications, enable open telemetry for tracing and realize cost control in a lifespan of a particular request.

Per-request keys are generated by AI DIAL Core, when it is making a request to the application and is valid only during the lifetime of this particular request from the Core to the application.

How It Works​

  • Let's consider that AI DIAL Core can receive an initial request from AI DIAL Chat or any of its clients. Such request we will call root requests. Example: AI DIAL Chat --> AI DIAL Core
  • AI DIAL Core can perform a series of requests to realize the initial request - let's call them non-root requests. Example: AI DIAL Chat --> AI DIAL Core --> Application A --> AI DIAL Core --> Application B...

Root Request:

  1. AI DIAL Core checks that the caller (chat user or application) has enough permissions and limits (based on the configuration) to perform the request and generates a new per-request key specifically for this request.
  2. AI DIAL Core stores an additional data associated with the per-request key either in memory or in Redis. The data may include but is not limited to: JWT of the user or API key of the application, trace-id, core-parent-span-id, list of the user files attached to the request.
  3. AI DIAL Core passes the per-request key to the application or model adapter in the API-key header. The application or model adapter then send the per-request key back to AI DIAL Core using the same API-key header for model requests or file storage API requests.
  4. AI DIAL Core invalidates the per-request key when the request is completed.

Non-root Requests:

  1. For all non-root requests, AI DIAL Core checks, that the incoming per-request key is valid and known (the one from the root request).
  2. It checks the permissions of the caller and then generates and saves a new outgoing per-request key. This key is also associated with specific data: all data from the root request (its parent request) and the necessary data for the current request.
  3. AI DIAL Core passes this new outgoing per-request key in the request header. The application or model adapter then send the per-request key back to AI DIAL Core using the same API-key header for model requests or file storage API requests.
  4. AI DIAL Core invalidates the outgoing per-request key when the request is completed.

Calculating Costs & Limits​

Per-request keys can be used to attribute costs and limits incurred by a request to a model to the user or application that initiated the request.

To attribute costs and limits to the request originator, traceparent should be included in the request header. trace-id is saved throughout the lifespan of the request in memory or in Redis.

Gathering Statistics​

For calculating statistics, AI DIAL Core uses trace-id and core-parent-span-id that are stored in the additional data linked to the per-request key.

Files Sharing​

Initially, AI DIAL Core verifies the permissions and limits of the request originator using the JWT or API Key associated with the initial request. It then generates a per-request key and links specific user files to it. Throughout the duration of the per-request key and across the entire call stack, AI DIAL Core uses this associated data to determine access rights for the application. If the application attempts to share a file which is not in the list of files associated with the per-request key or directly accessible to this application, the request results in 403 Forbidden error.

To share the output files with the user, AI DIAL Core grants a full access to the authorized application to a specific output folder in the user's bucket.

To provide the path of the folder for output files, we add appdata field to the response for the GET /v1/bucket. If this request is made with a per-request key the response will contain appdata with the path to the shared folder, including the user's bucket and the correct deployment-id for the application as registered in the AI DIAL Core configs.

{
  "bucket": "{application-bucket-id}",
  "appdata": "{user-bucket-id}/appdata/{deployment-id}",
}

Telemetry Tracing​

For tracing open telemetry, traceparent should be included in the request header. The open telemetry tracing does not interfere with the limits, statistics or file sharing.